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Abstract: 

Inspection  of  structures,  systems  and  engines  is  an  important  part  of  ensuring  continued  airworthiness  of  the 
civil  aircraft  fleet.  This  paper  describes  the  airworthiness  assurance  system  and  considers  applicable  bodies  of 
knowledge  which  help  understand  and  predict  aircraft  inspection  performance.  Two  examples  of  recent 
studies  of  aircraft  inspectors  are  used  to  illustrate  the  extra  depth  and  breadth  of  understanding  available 
where  such  knowledge  is  applied  to  these  tasks.  It  is  concluded  that  perhaps  we  have  two  separate  roles:  to 
predict  performance  and  to  improve  it.  Quantitative  prediction  will  never  be  complete,  but  better  estimates  of 
inspector  variability  help  us  set  more  realistic  inspection  intervals.  However,  for  improving  aircraft  inspection 
tasks  we  should  concentrate  on  broader  contextual  factors,  despite  our  inability  to  quantify  some  of  these 
effects. 

Human  Reliability  Issues  in  Aircraft  Inspection: 

Civil  Aviation  is  growing  at  over  3.34%  per  year,  and  the  total  annual  passengers  in  many  developed  countries 
is  comparable  to  the  country’s  population.  Flying  is  a relatively  safe  activity,  but  one  whose  failures  are 
dramatic  and  highly  publicized.  Thus,  the  exposure  to  risk  is  seen  by  the  population  as  high  and  increasing.  As 
airline  growth  increases,  the  prediction  (Boeing,  1997)  is  for  increasing  crashes,  up  to  one  per  week  in  2015 
unless  the  current  incident  rate  is  decreased. 

Of  the  most  visible  crashes,  known  as  hull-loss  accidents,  the  fraction  with  maintenance  or  inspection  as  a 
contributing  factor  has  been  about  20%  historically  (Boeing  1997),  but  the  rate  has  been  increasing  in  recent 
years.  Thus,  inspection  and  maintenance  errors  have  been  seen  recently  as  a major  airworthiness  emphasis 
(e.g.  Gore  Commission  Report,  1997). 

This  paper  considers  one  aspect  where  human  reliability  plays  a crucial  role,  that  of  inspection.  The  work 
reported  here  is  the  outgrowth  of  several  initiatives  by  regulatory  bodies,  primarily  the  Federal  Aviation 
Administration  (FAA)  in  the  USA,  Transport  Canada  and  the  UK’s  Civil  Aviation  Administration  (CAA). 
These  range  from  reliability  measurement  of  inspection  tasks  to  the  use  of  Crew  Resource  Management 
(CRM)  techniques  in  maintenance  and  inspection  activities.  The  aim  of  this  paper  is  to  consider  the  findings 
of  these  initiatives  and  other  applicable  human  factors  knowledge  in  the  domain  of  aviation  maintenance. 
What  can  each  contribute  to  improving  system  reliability?  What  are  lessons  for  other  highly-regulated  safety- 
critical  systems  which  have  an  inspection  component?  To  do  this,  we  first  present  an  overview  of  the  system 
for  inspecting  and  maintaining  aircraft,  and  then  summarize  the  findings  from  contributing  fields. 

The  System: 

Aiiworthincss  of  civil  aircraft  depends  upon  a process  by  which  a team  composed  of  aircraft  manufacturers, 
regulators  and  one  or  more  airlines  predict  possible  system  failures.  This  process.  Maintenance  Steering 
Group  3 or  MSG-3,  considers  possible  failure  pathways  (for  example  in  structures,  controls,  avionics)  and  for 
each  pathway  determines  a recovery  strategy.  For  structural  failure,  this  may  be  replacement  after  a fixed 
service  life,  regular  inspection  to  assure  detection,  or  an  indication  to  crew  of  the  malfunction.  In  this  paper 
the  concern  is  with  the  reliability  of  the  primary  failure  recovery  system  for  aircraft  structural  inspection: 
regular  inspection  to  assure  detection. 


Paper  presented  at  the  RTO  HFM  Workshop  on  “The  Human  Factor  in  System  Reliability  - Is  Human 
Performance  Predictable?  ",  held  in  Siena,  Italy,  1-2  December  1999,  and  published  in  RTO  MP-032. 
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Failure  modes  of  aircraft  structures  can  be  cracks,  corrosion,  fastner/bonding  failure  or  deformation  beyond 
the  plastic  limit.  Inspection  systems  are  designed  to  detect  all  of  these  in  a timely  manner,  i.e.  before  the 
failure  has  a catastrophic  effect  as  structural  integrity.  For  example,  crack  growth  rates  can  be  predicted 
probabilitistically  from  material  properties  and  applied  stresses,  so  that  the  MSG-3  process  can  schedule 
inspections  before  a potential  crack  becomes  dangerous.  However,  the  detection  system  has  certain  limits  on 
size  crack  that  can  be  detected,  so  that  MSG-3  typically  schedules  several  inspections  between  the  time  the 
crack  becomes  detectable  and  the  time  it  becomes  dangerous.  If  too  many  inspections  are  scheduled,  the  costs 
are  driven  up  in  a highly-competitive  industry,  and  the  risk  of  collateral  damage  is  increased  due  to  the 
inspection  process  itself.  Conversely,  if  too  few  inspections  are  scheduled,  the  probabilistic  rate  of  the  crack 
growth  prediction  process  may  combine  with  the  probabilistic  nature  of  the  detection  process  to  cause 
dangerous  cracks  to  remain  undetected.  Spectacular  failures  of  this  inspection  process  have  occurred  both  for 
aircraft  structures  (Aloha  incident,  Hawaii  1988)  and  engine  components  (Pensacola  incident,  Florida  1997). 

The  MSG-3  process  thus  requires  quantitative  data  on  inspection  reliability  to  function  correctly.  In  addition, 
no  rule-based  prediction  system  can  foresee  all  possible  malfunctions,  so  that  once  an  aircraft  is  in  service, 
regular  detailed  inspections  are  made  of  the  whole  structure  to  discover  any  unexpected  cracks.  When  such 
“new”  cracks  are  found,  the  information  is  typically  shared  between  manufacturers,  operators  and  regulators 
in  the  form  of  supplementary  inspections.  Similar  considerations  apply  to  other  failure  modes  such  as 
corrosion. 

This  whole  reliability  assurance  process  thus  rests  upon  an  inspection  system  w'hich  checks  both  points  where 
malfunctions  are  expected  and  points  where  they  are  not  expected,  for  a variety  of  malfunctions.  For  good 
reasons,  human  inspectors  are  part  of  this  inspection  system,  so  that  human  inspection  reliability  is  an 
essential  element  in  ensuring  structural  integrity,  and  hence  airworthiness.  The  rest  of  this  paper  considers 
bodies  of  knowledge  and  data  from  three  sources  which  should  be  applicable  to  human  inspection  reliability. 
Parts  of  this  material  have  been  reviewed  previously  (Drury  and  Spencer,  1997)  to  which  the  reader  is  referred 
for  further  details  and  references. 

The  Inspection  Task: 

The  inspection  task  implied  above  combines  two  goals:  detection  of  expected  malfunctions  and  detection  of 
unexpected  malfunctions.  Neither  detection  is  particularly  easy  or  particularly  rapid,  so  that  inspection  can  be 
a difficult  and  time-consuming  task.  In  some  ways  inspection  can  be  classified  as  an  ill-structured  task 
(Wenner,  1999)  because  there  is  no  simple  step-by-step  procedure  which  will  ensure  success,  and  because 
there  is  usually  no  knowledge  of  task  success  available  during  the  task.  Finding  (n)  malfunctions  in  a structure 
still  leaves  an  unknown  number  (hopefully  zero)  potentially  undetected. 

In  addition,  inspection  is  typically  scheduled  at  the  beginning  of  an  aircraft's  maintenance  visit  so  that 
malfunctions  can  be  detected  early  and  their  repair  scheduled  to  overlap  in  time  with  other  maintenance 
activities.  As  airlines  streamline  their  parts  inventory  to  reduce  holding  costs,  the  lead  time  for  replacement 
components  can  increase,  again  pressuring  the  inspection  system  to  ensure  early  detection.  Aircraft  typically 
arrive  following  scheduled  service,  i.e.  after  the  last  flight  of  the  day.  Following  opening  up  and  cleaning 
processes,  maximum  inspection  resources  are  committed  to  the  initial  inspection.  In  practice  this  means 
inspectors  working  overtime,  even  double  shifts,  starting  with  a night  shift,  under  some  implied  pressure  for 
early  detection.  Human  inspection  reliability  may  not  be  optimal  under  these  conditions. 

The  inspection  task  itself  is  classified  in  aviation  as  either  Visual  Inspection  or  non-destructive  inspection. 
Regulatory  bodies  have  issued  formal  descriptions  of  both  of  these  tasks  (e.g.  Bobo  (1989)  for  the  FAA),  and 
both  have  somewhat  different  characteristics  in  aviation 

Non-destructive  inspection  (NDI)  comprises  a set  of  techniques  to  enhance  the  ability  to  detect  small  and/or 
hidden  malfunctions.  One  set  of  NDI  techniques  are  those  which  enhance  what  is  essentially  still  a visual 
inspection  task,  for  example  X-ray,  fluorescent  particle,  magnetic  particle  or  D-sight.  They  show  cracks  which 
are  very  small  (fluorescent  particle)  or  hidden  within  other  structures  (X-ray).  Apart  from  the  steps  necessary 
to  ensure  a good  image,  they  have  many  of  the  human  interface  characteristics  of  visual  inspection.  The  other 
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set  of  NDI  techniques  are  focused  on  specific  malfunctions  in  specific  locations,  e.g.  eddy  current,  ultrasound. 
For  this  reason,  they  are  only  useful  for  detection  of  malfunctions  already  predicted  to  exist.  In  practice,  such 
NDI  techniques  are  much  more  proceduralized  than  visual  inspection  or  NDI  techniques  which  contain  a 
human  visual  inspection  component. 

Visual  inspection  is  much  more  common,  comprising  80%  of  all  inspection  Goranson  and  Rogers  (1983).  It 
consists  of  using  the  inspector's  eyes,  often  aided  by  magnifying  lenses  and  supplementary  lighting,  as  the 
detection  device.  Inspectors  must  visually  scan  the  whole  structure  of  interest,  typically  using  portable  mirrors 
to  examine  areas  not  directly  visible.  Whether  the  task  is  categorized  as  Visual  Inspection  or  NDI,  its  aim  is  to 
detect  flaws  (indications)  before  they  become  hazardous.  Next  we  consider  the  bodies  of  knowledge 
potentially  applicable  to  aircraft  inspection  reliability.  This  section  is  adapted  from  Drury  and  Spencer  (1997). 

Applicable  Knowledge  1.  NDI  Reliability: 

Over  the  past  two  decades  there  have  been  several  studies  of  human  reliability  in  aircraft  structural  inspection 
(Rummel,  Hardy  and  Cooper,  1989;  Spencer  and  Schurman,  1995;  Murgatroyd,  Worrall  and  Waites,  1994). 
All  of  these  to  date  have  examined  the  reliability  of  Non-Destructive  Inspection  (NDI)  techniques,  such  as 
cddy-currcnt  or  ultrasonic  technologies. 

From  NDI  reliability  studies  have  come  human/  machine  system  detection  performance  data,  typically 
expressed  as  a Probability  of  Detection  (PoD)  curve,  e.g.  Spencer  and  Schurman  (1 995).  This  curve  expresses 
reliability  of  the  detection  process  (PoD)  as  a function  of  a variable  of  structural  interest,  e.g.  crack  length, 
providing  in  effect  a psychophysical  curve  as  a function  of  a single  parameter.  Sophisticated  statistical 
methods  (e.g.  Hovey  and  Berens,  1988)  have  been  developed  to  derive  usable  PoD  curves  from  relatively 
sparse  data.  Because  NDI  techniques  are  designed  specifically  for  a single  fault  type  (e.g.  cracks),  and  much 
of  the  variance  in  PoD  can  be  described  by  just  crack  length,  the  PoD  is  a realistic  reliability  measure.  It  also 
provides  the  planning  process  with  exactly  the  data  required,  as  remaining  structural  integrity  is  largely  a 
function  of  crack  length. 

Both  the  FAA  ( National  Aging  Aircraft  Research  Program  Plan , 1993,  p.  26,  p.  35)  and  the  Air  Transport 
Association  (AT A)  have  recognized  the  need  for  equivalent  studies  of  the  reliability  of  visual  inspection  as  a 
research  priority. 

Applicable  Knowledge  2.  Industrial  Inspection: 

Human  factors  analyses  of  inspection  tasks  have  been  published  since  the  1050’s  and  1960’s  with  a steady 
evolution  of  approaches.  Early  studies  (e.g.  Thomas  and  Seaborne,  1961)  tended  to  be  rich  and  holistic 
descriptions  of  inspection  tasks.  They  focused  on  some  of  the  unique  perceptual  cues  used  by  experienced 
inspectors.  These  showed  for  example  that  inspectors  organize  their  perspectives  so  as  to  enhance  subtle  task 
relevant  visual  or  auditory  cues  and  surpress  what  a novice  would  perceive  as  salient  cues.  This  tradition  of 
description  has  occasionally  resurfaced  (Biederman  and  Shiffar,  1987;  Dalton,  1991)  but  has  been  largely 
replaced  by  more  quantitative  studies. 

The  next  wave  of  work  measured  human  performance  in  a variety  of  inspection  tasks,  typically  in  terms  of  the 
two  possible  errors:  missed  defects  and  false  alarms.  Reviews  of  this  work  are  readily  available  (Drury,  1992; 
Megaw,  Alexander  and  Richardson,  1979).  Table  1 classifies  some  of  the  factors  found  to  affect  inspection 
performance,  using  ICAO’s  SHELL  model  (ICAO,  1989).  Following  such  studies,  and  indeed  overlapping 
them,  were  model-oriented  studies  treating  inspection  as  either  a signal  detection  task  (Harris,  1969;  Drury 
and  Addison,  1973)  or  a visual  search  task  (Kundel;  1975;  Drury,  1990).  The  advantage  of  such  approaches  is 
that  they  can  use  the  underlying  models  to  predict  which  variables  are  most  and  least  likely  to  affect 
inspection  performance.  They  also  allow  succinct  descriptions  of  tasks  and  task  performance,  potentially 
leading  to  quantitative  models.  For  example,  NDI  studies  of  aircraft  inspection  often  provide  Relative 
Operating  Characteristic  (ROC)  curves  relating  miss  rate  to  false  alarm  rate  for  a given  defect  type. 
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An  inspection  model  combining  search  and  decision  (Drury,  1975)  can  also  be  helpful  in  understanding  the 
inspector’s  tasks  in  inspection.  This  model,  summarized  in  Figure  1,  shows  an  inspector  searching  an  item  by 
repeated  fixations  of  small  areas.  If  an  indication  (potential  defect)  is  found,  a decision  task  takes  place  to 
determine  whether  the  indication  should  be  classed  as  a reject.  If  not,  or  if  the  fixation  found  no  indications, 
search  continues.  The  inspection  task  stops  (or  moves  to  the  next  item)  when  there  is  no  further  time  left  for 
inspection,  either  because  of  the  inspector’s  stopping  policy  or  external  pacing  of  inspection.  This  model 
allows  us  to  specify  the  variables  affecting  each  stage.  Thus,  peripheral  visual  acuity  should  affect  fixation 
area  and  thus,  search  performance  (Courtney,  1984).  Conversely,  the  decision  stage  should  be  affected  by  cost 
and  probabilities  of  the  decision  outcomes  (Chi  and  Drury,  1998).  Overall,  this  model  has  been  useful  in 
interpreting  the  speed/  accuracy  tradeoff  in  inspection  (Drury,  1994). 


Figure  1 : Model  of  inspection  performance  incorporating  search  and  decision 


Knowledge  of  how  people  perform  inspection  tasks  in  both  manufacturing  industries  and  medical  diagnosis 
has  been  reviewed  many  times  (e.g.  Drury,  1997).  It  has  also  been  interpreted  in  an  aviation  context  following 
the  Aloha  incident  (Drury,  1989;  Wiener,  1989).  Briefly,  inspection  is  composed  of  several  functions  or 
processes,  the  most  error-prone  of  which  are  search  and  decision.  In  search,  the  inspectors’  eyes  (or  probe  for 
NDI)  move  around  the  area  to  be  inspected,  stopping  when  some  indication  is  found.  In  decision,  this 
indication  is  compared  to  known  (available  or  remembered)  standards  to  determine  whether  or  not  a 
reportable  fault  condition  exists. 

A flavor  of  the  findings  of  this  tradition  can  best  be  given  through  ICAO’s  model  of  human  factors  in 
aviation:  SHELL  (ICAO,  1989),  each  element  of  which  represents  a key  component  of  the  human/  machine 
system.  How  each  component  interfaces  with  the  individual  considered  (pilot,  air  traffic  controller,  AMT, 
inspector)  determines  the  sources  of  both  successful  human  performances  and  human  errors.  Table  1 
summaries  industrial  inspection  findings  using  this  aviation-based  model  of  human  factors. 
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Table  1:  Summary  of  inspection  findings  using  ICAO’s  SHELL  Model 


SHELL  System  Component 

Typical  Findings  from  Inspection  Studies 

S:  Software 

c.g.  procedures,  instructions, 
workcards,  feedback 

Instructions  given  to  the  inspector  have  a great  effect  on  both  p 
(detect)  and  p (false  alarm).  In  addition,  feedback  information  to 
the  inspector  has  large  positive  influences  on  performance 
(Gramopadhye  et  al,  1997). 

H:  Hardware 

e.g.  job  aids,  enhanced  vision 
systems,  magnifiers 

Equipment  such  as  semi-automated  visual  inspection  systems 
improve  performance  when  well-integrated  with  human  functions 
(Houetal,  1993). 

Enhanced  vision  systems,  such  as  magnification  or  lighting  aids 
sometimes  help,  sometimes  do  not.  Providing  visible  comparison 
standards  improves  decision. 

E:  Environment 

e.g.  lighting,  thermal,  noise 

Some  effects,  but  only  at  relatively  extreme  values  and  with  long 
exposure  times. 

L:  Liveware  (Individual) 

e.g.  individual  inspector 
characteristics 

Some  general  characteristics  of  “good”  inspectors,  such  as  field 
independence  and  peripheral  visual  acuity.  Often  each  inspection 
task  shows  performance  correlations  with  different  individual 
characteristics. 

L:  Liveware/Liveware 

e.g.  interactions  with  other  people 
in  system 

Job  design  is  important.  Inspectors  tend  to  feel  their  jobs  isolate 
them  from  others.  Expectations  of  others  can  have  large  effects  on 
what  gets  reported  as  fault. 

Applicable  Knowledge  3.  Human  Factors  in  Aviation  Operations: 

There  is  a long  tradition  of  human  factors  analysis  of  both  the  tasks  involved  in  flying/guiding  aircraft,  and  the 
accidents  arising  from  these  tasks.  Indeed,  one  of  the  earliest  human  factors  studies  (Fitts  and  Jones,  1947) 
analyzed  460  “pilot  error”  accidents  and  found  that  many  were  induced  by  poor  design  or  placement  of 
controls  and  displays  in  the  cockpits  of  the  time.  Over  the  succeeding  years,  these  studies  have  led  to  great 
improvements  in  the  design  of  cockpits,  selection  procedures  and  pilot  training  programs  (Wiener  and  Nagel, 
1988). 

In  recent  years  the  interest,  both  on  the  flight  deck  and  in  air  traffic  control,  has  focused  on  the  two  issues  of 
automation  and  interpersonal  interactions.  Automation  studies,  again  both  of  how  tasks  should  be  performed 
and  the  accidents  arising  when  they  arc  performed  incorrectly,  have  led  to  changes  in  automation  systems  (c.g. 
Phillips,  1998). 

Interpersonal  relations  on  the  flight  deck  have  also  been  studied  both  analytically  and  through  accident 
analysis.  From  this,  work  has  emerged  a body  of  theory  and  practice  known  generally  as  Resource 
Management.  Crew  Resource  Management  (CRM)  is  now  a regular,  and  potentially  ICAO  mandated, 
component  of  flight  training  and  retraining  programs  (e.g.  Heimreich,  Foushee,  Benson,  and  Russini,  1986; 
Foushee  and  Heimreich,  1988).  Pilots  (and  others)  learn  techniques  for  working  together  more  effectively  from 
flight  planning  through  to  handling  of  unplanned  incidents.  Such  results  have  found  rational  applicability  in  the 
aviation  maintenance  domain,  now  becoming  known  as  Maintenance  Resource  Management  (MRM).  Taylor 
(1991)  has  taken  a socio-technical  systems  approach  to  analysis  of  inter-personal  activities  in  maintenance.  This 
has  led  to  training  programs  (e.g.  Robertson,  1996;  Komamiski,  Russell  and  Johnson,  1996)  which  have  been 
successful  in  changing  attitudes  and  behaviors  of  maintenance  personnel. 
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Using  Applicable  Knowledge: 

Aircraft  inspection  has  already  benefited  from  some  of  these  knowledge  areas.  Thus  the  EC R IRE  program 
(Spencer  and  Schurman,  1995)  examined  one  NDI  technique,  eddy-current  inspection,  incorporating  human 
factors  variables.  They  were  able  to  test  one-person  versus  two  person  teams  (no  consistent  effects)  and  gross 
body  posture  (a  small  decrease  in  detection  performance  when  the  inspector  had  to  work  at  about  knee 
height).  A FAA  program  on  human  factors  in  aviation  maintenance  and  inspection  (c.g.  Drury.  Shepherd  and 
Johnson,  1997)  has  had  some  success  in  improving  documentation  design,  lighting  and  communications.  This 
program  expanded  the  search-plus-decision  model  following  industrial  inspection  findings  to  include  five 
generic  inspection  functions  (Drury,  1992): 

inspection,  e.g.  calibration,  documentation 
area  to  be  inspected,  e.g.  by  removing  access  hatches 
area  by  successive  fixations  or  probe  movements 
on  whether  indication  exceeds  standard 
by  signing  inspection  as  complete  or  recording  defect. 

Such  a task  description  invites  task  analysis,  which  would  lead  naturally  to  human  reliability  analysis  (HRA). 
Indeed,  perhaps  the  earliest  work  in  this  field  applied  HRA  techniques  to  construct  fault  trees  for  aircraft 
structural  inspection  (Lock  and  Strutt,  1985).  The  HRA  tradition  lists  task  steps,  such  as  expanded  versions  of 
the  generic  functions  above,  lists  possible  errors  for  each  step,  then  compiles  performance  shaping  factors  for 
each  error.  Such  an  approach  was  tried  early  in  the  FAA's  human  factors  initiative  (Drury,  Prabhu  and 
Gramopadhye,  1990),  but  was  ultimately  seen  as  difficult  to  use  because  of  the  sheer  number  of  possible 
eiTors  and  PSF's.  It  is  occasionally  revised,  e.g.  in  the  current  FRANCIE  project  (Haney,  1999)  using  a much 
expanded  framework  that  incorporates  inspection  as  one  of  a number  of  possible  maintenance  tasks.  Other 
attempts  have  been  made  to  apply  some  of  the  richer  human  error  models  (e.g.  Reason,  1990;  Hollnagel, 
1997;  Rouse,  1985)  to  inspection  activities  (Latorella  and  Drury,  1992;  Prabhu  and  Drury,  1992;  Latorella  and 
Prabhu,  1998)  to  inspection  tasks.  These  have  given  a broader  understanding  of  the  possible  errors,  but  have 
not  helped  better  define  the  PoD  curve  needed  to  ensure  continuing  airworthiness  of  the  civil  air  fleet. 

Two  Recent  Studies: 

To  help  understand  how  human  factors  can  contribute  to  the  domain  of  aircraft  inspection,  two  examples  are 
given.  The  first  pursues  an  analytical  approach  based  on  a task  breakdown,  while  the  second  examines  broader 
issues  affecting  human  reliability. 

The  first  study  was  the  Visual  Inspection  Research  Program  (VIRP)  undertaken  for  the  FAA  using  a retired 
Boeing  737  test  aircraft  at  Sandia  National  Laboratories  (Drury  and  Spencer,  1997).  Twelve  experienced 
airline  inspectors  performed  ten  different  inspection  tasks,  nine  on  the  aircraft  and  one  on  a series  of  fuselage 
test  panels  containing  known  cracks.  The  total  experiment  lasted  1.5  to  2 days  per  inspector  and  was 
performed  under  highly  realistic  conditions  in  a flight  hangar.  Overall,  performance  was  quite  variable. 
Inspectors  took  from  7.5  to  12.3  hours  of  inspection  time  for  the  ten  tasks.  On  a set  of  large  cracks  and 
corrosion  defects  which  the  manufacturers  would  expect  inspectors  to  find,  the  probability  of  detection  was 
also  quite  variable.  PoD  ranged  from  0.5  to  1.0  on  large  cracks  and  from  0.3  to  0.6  on  large  corrosion  areas. 
There  was  little  evidence  of  a speed/accuracy  tradeoff  across  inspectors.  There  were  also  low  correlations 
between  inspector  performance  on  the  10  tasks,  and  also  between  pre-test  measures  and  task  performance. 
Individual  differences  were  large  and  inconsistent. 

A more  detailed  analysis  of  this  data  is  possible  by  classifying  errors  into  search  errors  and  decision  errors. 
Drury  and  Sinclair  (1983)  showed  that  this  was  possible  in  industrial  inspection  of  aircraft  bearings.  For  the 
panel  inspection  task,  we  used  video  tape  records  to  classify  the  errors.  It  was  possible  to  see  from  the  video 
tape  whether  an  inspector  passed  quickly  over  a crack  defect  (search  error)  or  whether  he  paused  to  examine 
the  defect  more  closely  before  either  reporting  it  or  moving  on.  This  latter  was  a decision  error,  either  a miss 
or  as  false  alarm. 


Initiate 

Access 

Search 

Decision 

Response 


3-7 


Figure  2 shows  the  individual  differenees  between  inspectors  for  search  performance.  Note  that  probability  of 
search  success  is  rather  uniform  and  low.  The  mean  was  0.5  and  the  coefficient  of  variation  was  0.2.  For 
decision  performance,  Figure  2 shows  individual  inspector  results  plotted  on  Relative  Operating  Characteristic 
space.  The  variability  is  readily  apparent,  with  mean  performance  as  follows: 

p (correct hit):  mean- 0.84  CV_1.2 

p (correct  No):  mean  = 0.64  CV  = 1.0 


precision  Correct  No) 


Figure  2:  Relative  Operating  Characteristic  (ROC)  curve  for  decision  component  in 
aircraft  structural  inspection.  Each  point  is  from  one  inspector. 

Thus,  search  performance  could  be  characterized  as  consistently  poor,  whereas  decision  performance  was 
better,  but  highly  variable.  Search  and  decision  performance  were  statistically  unrelated.  Such  findings  allow 
us  to  focus  interventions,  for  example  by  improving  lighting  and  Raining  to  support  search,  while  using 
training  and  feedback  to  reduce  intcr-inspcctor  variability  in  decision  (Gramopadhyc,  Drury  and  Prabhu, 
1993). 

The  second  experiment  was  similar  to  VIRP,  but  performed  on  a commuter  aircraft  (Fairchild  Metro)  using 
experienced  regional  airline  inspectors.  This  study  had  inspectors  perform  seven  inspection  tasks,  again  over 
1.5  to  2 days.  Performance  was  again  highly  variable  between  inspectors  and  between  tasks  with  almost  no 
correlations  between  task  perfoimance  or  between  task  performance  and  pre-tests.  However,  in  this 
experiment,  the  major  concentration  was  on  the  subtleties  of  the  inspection  task  and  its  context,  detailed  in 
Wenner  and  Drury  (1997). 
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